{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "35871b6a",
   "metadata": {},
   "source": [
    "# task0：自然语言处理之PyTorch情感分析简介\n",
    "\n",
    "> 注：本期组队学习仅适用于需要PyTorch 1.8或以上版本的torchtext 0.9或以上版本。如果你使用的是torchtext 0.8，请点击[这里](https://github.com/bentrevett/pytorch-sentiment-analysis/tree/torchtext08)。\n",
    "\n",
    "本期学习使用的软件和版本为：[Pytorch1.8](https://github.com/pytorch/pytorch)，[torchtext0.9](https://github.com/pytorch/text) 和Python3.7\n",
    "\n",
    "本期组队学习主要从一下几个方面进行学习：\n",
    "\n",
    "- 利用RNN进行情感二分类\n",
    "- 利用RNN的各种变体，如LSTM, BiLSTM等进行情感二分类\n",
    "- 利用更快的模型FastText进行情感二分类\n",
    "- 利用CNN进行情感二分类\n",
    "- 情感多分类\n",
    "- 利用BERT进行情感分类\n",
    "    \n",
    "前两个Text将介绍情感分析的常用方法：递归神经网络（RNN）；第三个Text介绍了[FastText](https://arxiv.org/abs/1607.01759)模型；最后一个task的学习覆盖一个[卷积神经网络](https://arxiv.org/abs/1408.5882)（CNN）模型。\n",
    "\n",
    "还有两个额外的“附录”。第一部分介绍如何使用torchtext加载自己的数据集，第二部分简要介绍torchtext提供的经过预训练的单词嵌入。这部分自由学习，在组队学习中不做要求。\n",
    "\n",
    "## 环境配置\n",
    "\n",
    "①要安装Pytorch，请参阅[Pytorch网站](https://pytorch.org/get-started/locally)上的安装说明。\n",
    "\n",
    "②要安装torchtext，请执行以下操作：\n",
    "\n",
    "```bas\n",
    "pip install torchtext\n",
    "```\n",
    "\n",
    "③若安装速度较慢，可改为以下命令：\n",
    "\n",
    "```ba\n",
    "pip install -i https://pypi.tuna.tsinghua.edu.cn/simple torchtext\n",
    "```\n",
    "\n",
    "④此外，我们还将使用spaCy来标记数据。要安装spaCy，可以按照[spaCy官网](https://spacy.io/usage)的指令来安装，或者执行以下命令：\n",
    "\n",
    "```猛击\n",
    "python -m venv .env\n",
    ".env\\Scripts\\activate\n",
    "pip install -U pip setuptools wheel\n",
    "pip install -U spacy[transformers,lookups]\n",
    "python -m spacy download zh_core_web_sm\n",
    "python -m spacy download en_core_web_sm\n",
    "```\n",
    "\n",
    "⑤对于Taxt6，我们将使用transformers库，可以通过以下方式安装（更改为清华源）：\n",
    "\n",
    "```猛击\n",
    "pip install -i https://pypi.tuna.tsinghua.edu.cn/simple transformers\n",
    "```\n",
    "\n",
    "这些教程是使用的transformers版本为4.3。\n",
    "\n",
    "## 组队学习基本内容\n",
    "\n",
    "* task1- [情感分析baseline](https://github.com/datawhalechina/team-learning-nlp/blob/master/Emotional_Analysis/task1%20%EF%BC%9A%E6%83%85%E6%84%9F%E5%88%86%E6%9E%90baseline.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bentrevett/pytorch-sentiment-analysis/blob/master/1%20-%20Simple%20Sentiment%20Analysis.ipynb)\n",
    "\n",
    "这一章主要介绍PyTorch with torchtext项目的工作流。我们将学习如何：加载数据、创建训练/测试/验证拆分、构建词汇表、创建数据迭代器、定义模型以及实现训练/评估/测试循环。该模型将简单但是性能较差，可以将其看作一个Baseline，可以用于学习整个情感分析的处理过程，在后续教程中我们将对此模型进行改进。\n",
    "\n",
    "* task2-[Updated情感分析 ](https://github.com/datawhalechina/team-learning-nlp/blob/master/Emotional_Analysis/task2%EF%BC%9AUpdated%E6%83%85%E6%84%9F%E5%88%86%E6%9E%90%20.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bentrevett/pytorch-sentiment-analysis/blob/master/2%20-%20Upgraded%20Sentiment%20Analysis.ipynb)\n",
    "\n",
    "现在我们已经学习了情感分析的基本工作流程，下面我们将学习如何改进模型：使用压缩填充序列、加载和使用预先训练word embedding、不同的优化器、不同的RNN体系结构、双向RNN、多层（又称深层）RNN和正则化。\n",
    "\n",
    "* task3-[Faster情感分析](https://github.com/datawhalechina/team-learning-nlp/blob/master/Emotional_Analysis/task3%EF%BC%9AFaster%20%E6%83%85%E6%84%9F%E5%88%86%E6%9E%90.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bentrevett/pytorch-sentiment-analysis/blob/master/3%20-%20Faster%20Sentiment%20Analysis.ipynb)\n",
    "\n",
    "在我们介绍了使用RNN的升级版本的情感分析之后，我们将研究一种不使用RNN的不同方法：我们将实现论文 [《Bag of Tricks for Efficient Text Classification》](https://arxiv.org/abs/1607.01759)中的模型，该论文已经放在了教程中，感兴趣的小伙伴可以参考一下。这个简单的模型实现了与第二节中的升级的情感分析相当的性能，但训练速度要快得多。\n",
    "\n",
    "* task4-[卷积情感分析](https://github.com/datawhalechina/team-learning-nlp/blob/master/Emotional_Analysis/task4%EF%BC%9A%E5%8D%B7%E7%A7%AF%E6%83%85%E6%84%9F%E5%88%86%E6%9E%90%20.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bentrevett/pytorch-sentiment-analysis/blob/master/4%20-%20Convolutional%20Sentiment%20Analysis.ipynb)\n",
    "\n",
    "接下来，我们将介绍用于情绪分析的卷积神经网络（CNN）。该模型将是[Convolutional Neural Networks for Sentence Classification](https://arxiv.org/abs/1408.5882)的实现。\n",
    "\n",
    "* task5-[多模型融合情绪分析](https://github.com/datawhalechina/team-learning-nlp/blob/master/Emotional_Analysis/task5%EF%BC%9A%E5%A4%9A%E7%B1%BB%E5%88%AB%E6%83%85%E6%84%9F%E5%88%86%E6%9E%90.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bentrevett/pytorch-sentiment-analysis/blob/master/5%20-%20Multi-class%20Sentiment%20Analysis.ipynb)\n",
    "\n",
    "这一章，我们将使用包含以上两种模型的处理形式，这在NLP中很常见。我们将使用Text4中的CNN模型和一个包含6个分类的新数据集。\n",
    "\n",
    "* task6-[使用Transformers进行情感分析](https://github.com/datawhalechina/team-learning-nlp/blob/master/Emotional_Analysis/task6：Transformers情感分析.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bentrevett/pytorch-sentiment-analysis/blob/master/6%20-%20Transformers%20for%20Sentiment%20Analysis.ipynb)\n",
    "\n",
    "这一章，我们将学习如何使用transformers库加载预训练的transformer模型，实现论文[BERT：Pre-training of Deep Bidirectional Transfoemers for Language Understanding](https://arxiv.org/abs/1810.04805)中的BERT模型（该论文也以放入教程中），并使用它完成文本的embeddings。这些embeddings可以输入到任何模型中来预测情绪，在这里，我们使用了一个门循环单元（GRU）。\n",
    "\n",
    "## 拓展（待更新……）\n",
    "\n",
    "* A-[在自己的数据集上使用torchtext]() [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bentrevett/pytorch-sentiment-analysis/blob/master/A%20-%20Using%20TorchText%20with%20Your%20Own%20Datasets.ipynb)\n",
    "\n",
    "因为本教程使用的数据集为TorchText的内置数据集，附录A说明了如何使用TorchText加载自己的数据集。\n",
    "\n",
    "* B-[再看word embedding]() [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bentrevett/pytorch-sentiment-analysis/blob/master/B%20-%20A%20Closer%20Look%20at%20Word%20Embeddings.ipynb)\n",
    "\n",
    "通过使用TorchText提供的预训练word embedding来查看类似单词，以及实现一个简单的，完全基于word embedding的拼写错误校正器。\n",
    "\n",
    "* C-[加载、保存和固定Word embedding]() [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bentrevett/pytorch-sentiment-analysis/blob/master/C%20-%20Loading%2C%20Saving%20and%20Freezing%20Embeddings.ipynb)\n",
    "\n",
    "我们知道，在NLP领域，预训练语言模型已经发挥着越来越大的作用，在本附录中，我们将介绍：如何加载自定义单词嵌入，如何在训练我们的模型时固定和解除word embedding，以及如何保存我们学到的embedding，以便它们可以在其他模型中使用。\n",
    "\n",
    "## 参考资料\n",
    "\n",
    "* http://anie.me/On-Torchtext/\n",
    "* http://mlexplained.com/2018/02/08/a-comprehensive-tutorial-to-torchtext/\n",
    "* https://github.com/spro/practical-pytorch\n",
    "* https://gist.github.com/Tushar-N/dfca335e370a2bc3bc79876e6270099e\n",
    "* https://gist.github.com/HarshTrivedi/f4e7293e941b17d19058f6fb90ab0fec\n",
    "* https://github.com/keras-team/keras/blob/master/examples/imdb_fasttext.py\n",
    "* https://github.com/Shawn1993/cnn-text-classification-pytorch\n",
    "* https://github.com/bentrevett/pytorch-sentiment-analysis\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3aeaa46c",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
